Should we maybe have it a part of SDK?
Because we probably want to control the fps (e.g. in Safari 14 FPS => 15 FPS and its high) and there were a lot of pitfalls in it

I don't think we should abstract the entire navigator.mediaDevices.getUserMedia() call or the full video configuration - clients should be able to control and adjust these to their specific needs.

That said, since we do care about the frameRate (especially its max value) and possibly the max width/height and/or aspectRatio of the stream, we could expose these as helpers, verify the inputs, or cap them at our allowed limits - and add an error or warning if needed.

wdyt?

In terms of a backend I am torn between forcing max fps (i.e. dropping frames which arrive too early) or allowing any fps.
The former solution might be too strict (there is no problem if sometimes frames arrive a bit earlier or a bit late, but if we drop frames forcefully), hence we stick to the latter.